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Introduction 



This report summarizes the activities and findings of Horizon Research, Inc. (HRI) in its external 
evaluation of AAPT’s Physics Teacher Resource Agents (PTRA) Rural project since June 2004. 
During this period, from June 2004 to May 2005, HRI has: 

• Administered pre- and post-institute questionnaires to the PTRAs attending the 2004 
PTRA institute; 

• Observed a portion of the PTRA institute in Sacramento; 

• Conducted two focus group sessions with a total of 12 PTRAs who led the 2004 rural 
institutes; 

• Interviewed a random sample of 10 PTRAs who attended the 2004 PTRA institute; 

• Administered a questionnaire to all teachers who attended the 2004 rural institutes; 

• Assisted the project in the development of and administered (in a pre-test/post-test 
design) a kinematics, dynamics, energy, and momentum content assessment to 
outreach participants attending rural institutes focused on these topics; 

• Observed a sample of the 2004 rural institutes; and 

• Assisted the project in the development of and administered a student assessment 
focusing on kinematics, dynamics, energy, and momentum in a study of the impact of 
the AAPT/PTRA rural program on students. 

This report is divided into five main sections. The first provides an overview of the 
AAPT/PTRA Rural project and a description of the key questions guiding the evaluation. The 
second presents data on the 2004 PTRA institute, including PTRAs’ expectations for the 
institute, their perceptions of the quality of the professional development, and the impact of the 
institute on their preparedness to lead rural institutes. The third section reports data collected on 
the rural institutes held during the summer of 2004. These data include a description of the 25 
rural institutes and the teachers attending them, as well as feedback from the PTRAs leading 
these institutes. This section also reports the results of a study of the impact of the project on the 
content knowledge of outreach participants. The fourth section describes a study of the impact 
of the AAPT/PTRA Rural project on student achievement in kinematics, dynamics, energy, and 
momentum, which is currently in progress. The final section summarizes the key findings and 
presents HRTs recommendations for the project. 



Overview of the AAPT/PTRA Rural Project and Evaluation 

As stated in the grant proposal, the primary aim of the AAPT/PTRA Rural project is to “serve 
isolated and neglected rural teachers by building on the experience, expertise, and resources of 
the existing PTRA program. The project will provide opportunities for these teachers to grow 
professionally in physics content, in the use of technology for instruction, and in established 
teaching strategies. Additionally these teachers will develop into a professional and supportive 
network.” To accomplish these goals, the project has adopted a trainer-of-trainers approach. 
The first tier consists of the PTRAs, typically accomplished physics teachers. At a week-long 
PTRA institute, the PTRAs are trained to present workshops on a wide variety of physics topics 
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Most institute workshops are six-hours in length and focus on familiarizing the PTRAs with the 
classroom activities in the workshop manual. The institute also provides opportunities for the 
PTRAs to network and share ideas related to the classroom and to workshop leadership. The 
major goal for the summer institute is to provide the PTRAs with the knowledge and skills 
needed to effectively lead institutes for rural teachers. 

PTRA-led rural institutes, the second tier, are typically five days long and are intended to focus 
on one or two core physics topics (e.g., kinematics and dynamics). In addition, the project has 
included two, day-long follow-up workshops in the model. These workshops are intended to 
give the outreach participants an opportunity to revisit concepts and skills from the rural institute 
and to share and reflect on their efforts at incorporating what they learned into their classrooms. 

The rural institutes include a strong technology component, seeking to introduce outreach 
participants to a number of the tools that can be used to support physics instruction, including 
graphing calculators and calculator/computer-based laboratory activities. These institutes also 
give rural teachers, who are often the only science teacher in their school, an opportunity to 
network with other science teachers. At this second tier, the project expects to have an impact on 
rural teachers’ understanding of important physics content and their use of effective teaching 
strategies. Further, the project hypothesizes that these changes will lead to impacts in student 
learning. 

The evaluation plan for the AAPT/PTRA Rural project contains both formative and summative 
components and focuses on seven key questions: 

1 . How successful is the project at recruiting and maintaining a cadre of PTRAs, 
including teachers from the areas being served by the rural centers? 

2. To what extent does the PTRA institute prepare PTRAs with the physics and 
pedagogical content knowledge needed to present outreach workshops? 

3. To what extent does the PTRA institute prepare PTRAs with the leadership skills and 
professional development strategies that will enable them to design and implement 
extended high-quality professional development workshops that provide in-depth 
examination of physics content and standards-based teaching strategies? 

4. How successful is the project at initiating and maintaining the network of rural 
centers, including recruiting, training, and providing on-going support to each Rural 
Regional Coordinator? 

5. How successful is the project in reaching the goal of providing 108 hours of 
professional development (over three years) to under-served rural teachers and what 
is the quality of that professional development? 

6. What impacts does the project have on outreach participants’ attitudes, physics and 
pedagogical preparedness, and classroom practices? 
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7. What impact does teachers’ participation in the rural institutes have on their students’ 
achievement in physics? 

Although it is too early in the project to answer these questions fully, data collected during the 
project’s third year provide some insight into the project’s progress in reaching its goals. 



Preparation of the PTRAs: The 2004 PTRA Institute 

As noted above, the goal of the PTRA summer institute is to equip the PTRAs with the 
knowledge and skills necessary to provide high-quality, effective professional development for 
rural teachers. The skills and knowledge needed by the PTRAs include: 

• In-depth understanding of physics content; 

• Knowledge of, and experience using, effective physics teaching strategies; 

• Knowledge of effective professional development strategies/adult learning theory; and 

• Skill at designing and implementing high-quality professional development. 

The PTRA institute incorporates a variety of activities, including presentations by physics 
professionals, a session in which PTRAs share a favorite classroom activity or demonstration, 
and opportunities for networking. However, the main component of the institute is a set of 
workshops which focus on various physics topics, technological tools (e.g., graphing 
calculators), and/or teaching strategies. These workshops are developed by selected PTRAs, 
members of the project leadership, and/or other interested and knowledgeable members of the 
physics education community. Most of these workshops are six-hours long, though a few are 
three -hours in length. The workshops provide opportunities for the PTRAs to experience a 
sample of the classroom activities included in the workshop manual, and a forum to discuss 
physics content, classroom practices, and issues of leadership. 

In July of 2004, the project gathered 85 PTRAs in Sacramento, CA for the institute. This year, 
three new PTRAs attended the institute. The project offered 13 workshops during the 2004 
PTRA institute, covering topics such as kinematics, electricity, energy, laboratory interfacing 
devices, and graphical analysis. This section of the report focuses on the quality and impact of 
the PTRA institute using data collected from the pre- and post-institute questionnaires, evaluator 
observations, and interviews with PTRAs. 



The PTRAs 

The pre-institute questionnaire gathered a variety of data from the PTRAs, including 
demographic characteristics and information on their learning needs as professional development 
providers. Seventy PTRAs responded to the pre-institute questionnaire, a response rate of 82 
percent. Table 1 shows the demographic characteristics of the responding PTRAs. Sixty percent 
of the 2004 PTRAs were male; nearly all were white. About half taught in suburban schools, the 
rest were evenly split between rural and urban schools. Eighty-one percent taught physics and/or 
physical science during the 2003-04 academic year and two-thirds have over 20 years of 
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teaching experience. The majority of attendees became PTRAs prior to 1997; 16 percent have 
become PTRAs since the beginning of the rural project in 2002. 



Table 1 

Demographic Data for PTRAs Attending the 2004 Summer Institute 





Percent of Respondents 
(N = 70) 


Physics/Physical Science in Previous Year Teaching Assignment 


81 


Gender 




Male 


60 


Female 


40 


Race/Ethnicity 




White 


96 


Asian or Pacific Islander 


1 


American Indian or Alaskan Native 


1 


African-American 


0 


Hispanic 


0 


Other 


1 


Location of School 




Suburban 


52 


Urban 


24 


Rural 


24 


Year Originally Became a PTRA 




1985-1988 


27 


1992-1996 


36 


1997-2001 


21 


2002-2004 


16 


Membership in Professional Organizations 




AAPT 


96 


NSTA 


57 


Years of Physics/Physical Science Teaching Experience 




0-5 Years 


1 


6-10 Years 


9 


11-15 Years 


9 


16-20 Years 


14 


21-25 Years 


14 


26-30 Years 


24 


31-35 Years 


14 


36 or More Years 


14 



The Quality and Impacts of the PTRA Institute 

PTRAs ’ Needs and Expectations 

Knowing what participants’ needs and expectations are for a professional development 
experience can provide valuable insight into their perceptions of the quality of that experience. 
The pre-institute questionnaire asked PTRAs the extent to which additional training in a number 
of areas would help them become more effective professional development providers. As can be 
seen in Table 2, the PTRAs’ perceived a moderate to strong need for additional leadership 
training in many areas. About three-quarters of the PTRAs cited working with adults (i.e., 
research on how people learn, and principles of effective professional development facilitation) 
as an area in which they needed further training. Roughly 60 percent indicated needs in areas 
related to their own classroom practice (i.e. physics content and activities). Close to half of the 
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PTRAs saw little or no need for additional training in physics content. It is not surprising that 
learning physics content was not a major area of need, as many of the PTRAs are well-versed in 
physics. 



Table 2 

PTRAs’ Needs for Additional Leadership Training 





Percent of Respondents 
(N = 70) 




No to a 
little 

additional 

training 


Moderate 
amount of 
additional 
training 


A good deal to 
a lot of 
additional 
training 


Strategies for implementing the principles of effective professional 
development in workshops for other teachers 


21 


46 


33 


The research on common misconceptions/student thinking in physics 


21 


54 


25 


The research on the principles of effective professional development 


24 


47 


29 


The research on how people learn 


26 


40 


34 


Activities for physics instruction 


34 


47 


19 


Technologies for physics instruction 


39 


39 


22 


Physics content 


46 


40 


14 



The data on what the PTRAs hoped to gain from the institute contrasts with their professed 
needs. The most common response, given by 30 of the 69 PTRAs answering this open-ended 
question, was increasing their repertoire of classroom activities. Learning teaching strategies to 
use in their own classrooms and increasing their knowledge of physics content were also 
mentioned by many PTRAs (22 and 19, respectively). Only 17 PTRAs indicated that they hoped 
to learn new strategies for working with adult learners. These data indicate that many PTRAs 
entered the institute focusing on what would benefit them as a teacher rather than as a workshop 
leader. As three PTRAs wrote: 

[I hope to learn] new ideas and techniques for teaching physics and physical science. 

I expect to increase my content, what I teach and delivery, how I teach. 

[I hope to get] continued reinforcement in both content and teaching strategies. 

Data from the post- institute questionnaire^ show a similar pattern. When asked why they had 
selected to participate in the workshops they did, the most common response, given by 19 of the 
56 respondents, was that they selected workshops to increase their ability to teach the topic to 
their students. As three PTRAs wrote: 

They had ideas I could use in my classroom. 

I want to teach ] topic] this way to my students next year. 



* Fifty-six PTRAs returned the post-institute questionnaire, a response rate of 66 percent. 



Horizon Research, Inc. 



5 



May 2005 







[I was] looking for new ideas to teach students. 



Eighteen PTRAs said they chose their workshops because they are included in the rural institute 
sequence, and 15 stated that they selected workshops they had never taken before. One PTRA 
wrote: 



They contain important content, content of interest to all physical science/physics 
teachers in middle as well as high schools. I hoped to get new ideas/activities for my own 
teaching as well. 

These data from the pre- and post-institute questionnaires indicate that there was a mismatch 
between the PTRAs’ needs (i.e., training as professional development providers) and what they 
were hoping to get out of the PTRA institute (i.e., classroom activities and teaching strategies). 

PTRAs ’ Experience at the PTRA Institute 

The main vehicle for preparing the PTRAs as professional development providers is the institute 
workshops. The project offered 12 workshops during the 2004 PTRA institute, plus a three-hour 
leadership session. Table 3 shows the title, duration, and percentage of PTRAs taking each 
workshop. The workshop taken by the greatest number of PTRAs was CASTLE ( Capacitor- 
Aided System for Teaching and Learning Electricity), which was offered as a sequence of two, 
six-hour workshops. Having a large number of PTRAs trained in the CASLTE workshop should 
benefit the project as electricity is the core topic covered in the third year of the rural institutes’ 
three-year sequence. 



Table 3 



Participation Data for Workshops 
Offered during the 2004 PTRA Institute 





Duration 

(Hours) 


Percent of PTRAs Taking 
Workshop in 2004 
(N = 56) 


CASTLE (Electricity) 


12 


58 


Make and Take 


3 


49 


Leadership and Workshop Methods 


3 


48 


Gravity 


6 


45 


Interfacing - Vernier 


3 


44 


Energy - Societal Issues 


6 


43 


The Electromagnetic Spectrum (NASA workshop) 


6 


42 


Interfacing - PASCO 


3 


41 


Momentum and Impulse 


6 


39 


Energy 


6 


26 


Kinematics 


6 


26 


Graphical Analysis 


6 


20 


Newton’s T"** Law 


6 


20 
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Data collected from the PTRAs after the institute indicate that they considered the workshops to 
be of high quality. As can be seen in Table 4, a majority of PTRAs rated the quality of 
instruction in each workshop as high in quality; all responding PTRAs rated Graphical Analysis, 
Interfacing-Vernier, and Newton’s 2”“^ Law highly. 



Table 4 

PTRAs Rating Workshop Instruction as High Quality^ 





N* 


Percent of PTRAs 


Graphical Analysis 


11 


100 


Interfacing - Vernier 


22 


100 


Newton's Law 


11 


100 


Make and Take 


25 


96 


The Electromagnetic Spectrum 


21 


95 


Interfacing - PASCO 


21 


95 


CASTLE (Electricity) 


31 


94 


Kinematics 


13 


92 


Momentum and Impulse 


20 


90 


Energy-societal issues 


23 


87 


Gravity 


25 


84 


Energy 


14 


79 


Leadership and Workshop Methods 


25 


64 



^ Includes those who rated the item a 4 or 5 on a five-point scale from 1 “poor” to 5 “excellent.” 
* By design, not all PTRAs participated in each workshop; the total number responding for each 



workshop is included in the table. 



The PTRAs were also asked on the post-institute questionnaire and in interviews^ what aspects 
of the institute were particularly good. The quality of specific institute workshops was 
mentioned by 15 of the 49 respondents on the questionnaire and by 6 of the 10 interviewed 
PTRAs. As these two PTRAs said during interviews: 

I liked two things, being with the other PTRAs and the CASTLE workshop. I spent two 
days at the CASTLE workshop. At first I was not clear about where it was going and at 
the end of the first day I was on board, and now I want to try the materials here at [my 
school], and it will help me to lead workshops of that nature at other institutes. 

A couple of the break out sessions were good. . .Kinematics and Gravity were very good. 

Networking/sharing ideas with other PTRAs was also cited as being particularly good aspects of 
the institute. These aspects were mentioned by the majority (30 of 49) of the questionnaire 
respondents and by half of those interviewed. 



^ During the winter of 2004-05, HRI conducted telephone interviews with a random sample of 10 PTRAs who had 
attended the 2004 PTRA Institute. 
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Although only 64 percent of the participating PTRAs rated the Leadership workshop as high 
quality, the project may interpret this finding positively. This year, the leadership made a more 
concerted attempt to focus the PTRAs more on issues of leadership than on their own classroom 
instruction. A key component of this effort was a major revision to the Leadership workshop. In 
previous years, the session was devoted mostly to reviewing the various forms and record- 
keeping requirements for the outreach institutes. This year, the Leadership workshop included a 
much greater focus on issues related to professional development. 

The interviews with PTRAs asked about their opinions of the Leadership workshop. Seven out 
of 10 interviewed PTRAs had attended the workshop. Of these, four PTRAs indicated that they 
found the workshop to be helpful, with two PTRAs saying the workshop encouraged them to 
increase their emphasis on pedagogy in their outreach workshops. One PTRA stated that the 
session was informational about the vision of the program. The three who did not find the 
workshop helpful indicated that they thought too much time was still devoted to reviewing forms 
and procedures and wanted more time spent on improving their practices as professional 
development providers. 

Given the changes in the Leadership workshop, it is not surprising that the session was met with 
mixed reactions from the PTRAs. These changes may have forced some PTRAs out of their 
comfort zone by challenging them to reflect on their practices as professional developers rather 
than classroom teachers. However, these changes are likely to have a positive impact on the 
project in the long run, and provide a start that can be built upon in future institutes. 

In addition to asking the PTRAs about their perceptions of the quality of the summer institute 
workshops, the PTRAs were asked about the extent to which the institute focused on various 
goals. The data indicate that the institute focused heavily on what the PTRAs hoped to get (i.e., 
classroom strategies and activities) and less on what they indicated they needed to be more 
effective professional development providers. As can be seen in Table 5, 85 percent of the 
PTRAs indicated that the institute provided ample opportunities for getting new activities for 
physics instruction. Only 54 percent indicated that the institute focused heavily on effective 
professional development strategies. The research on common physics misconceptions, 
principles of effective professional development, and research on how people learn were 
mentioned by a minority of PTRAs (40, 33, and 26 percent, respectively). These areas that were 
less of a focus of the institute are the same areas in which a substantial proportion 
(approximately three-quarters) of the PTRAs prior to the summer institute indicated they needed 
a moderate to a great amount of additional training (see Table 2). 
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Table 5 

PTRAs’ Indicating that each of the 
Following Occurred^ at the Summer Institute 





Percent of Respondents 
(N = 56) 


Gained activities for physics instruction 


85 


Gained experience with technologies for physics instruction 


68 


Learned strategies for implementing the principles of effective professional development 




into workshops for other teachers 


54 


Learned physics content 


53 


Learned about the research on common misconceptions in physics 


40 


Learned about the research on the principles of effective professional development 


33 


Learned about the research on how people learn 


26 



Includes those who rated the item 4 or 5 on a five-point scale from 1 “not at all” to 5 “to a great extent.” 



These data are consistent with HRI’s observations at the summer institute. In the sessions HRI 
observed, the majority of the time was spent having the PTRAs work through classroom 
activities as if they were students. There were times during the workshops when the leaders 
discussed strategies for working with teachers in outreach workshops. However, the amount of 
attention to this topic varied widely among the workshops. In addition, these “discussions” often 
consisted of the leader sharing tips on how they facilitated an activity with little opportunity for 
the PTRAs to reflect on, or receive feedback about, their leadership skills. The PTRAs rated the 
quality of the workshops highly, as would be expected given that the PTRAs entered the institute 
primarily interested in improving their classroom practice. 

Impacts of the PTRA Institute 

By comparing responses from the pre- and post-institute questionnaires, HRI is able to examine 
the impact of the institute on the PTRAs’ perceptions of their preparedness to provide the 12 
workshops offered in the PTRA institute to outreach participants.^ For 5 of the 12 workshops at 
the institute, PTRAs participating in a workshop had significantly greater increases in their 
perceptions of preparedness to lead a workshop than non-participants (see Table 6). These 
workshops were: CASTLE (Electricity), The Electromagnetic Spectrum, Energy-Societal Issues, 
Gravity, Interfacing-PASCO, and Interfacing-Vernier. 

There were no significant differences in the change in preparedness of participants and non- 
participants for the other workshops. There are a number of reasons that may explain the lack of 
differences. For some workshops, many participants indicated on the pre-institute questionnaire 
that they were already well prepared in the workshop topic before attending, leaving little room 
for growth. Similarly, many of the non-participating PTRAs regarded themselves as well 
prepared in the workshop topics, thus reducing the likelihood of detecting a significant difference 
between participants and non-participants. For other workshops, the small number of 
respondents decreases the likelihood of detecting a significant change. 



^ HRI was able to match the pre- and post-questionnaire responses of 52 PTRAs; some PTRAs submitted one, but 
not both questionnaires. 
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Table 6 

PTRAs Feeling Well Prepared to Present Each 
of the Following Workshops, by Workshop Participation^ 





Percent of PTRAs 


N* 


Pre 


Post 


CASTLE (Electricity)* 








Participants 


30 


70 


97 


Non-Participants 


18 


72 


61 


The Electromagnetic Spectrum (NASA workshop)* 








Participants 


17 


82 


88 


Non-Participants 


28 


36 


25 


Energy 








Participants 


12 


25 


92 


Non-Participants 


36 


75 


78 


Energy - Societal Issues* 








Participants 


21 


29 


86 


Non-Participants 


28 


32 


21 


Graphical Analysis 








Participants 


10 


80 


100 


Non-Participants 


39 


72 


74 


Gravity* 








Participants 


21 


52 


95 


Non-Participants 


30 


67 


40 


Interfacing - PASCO* 








Participants 


18 


33 


72 


Non-Participants 


29 


28 


38 


Interfacing - Vernier* 








Participants 


19 


58 


95 


Non-Participants 


27 


48 


59 


Kinematics 








Participants 


13 


77 


92 


Non-Participants 


34 


97 


97 


Momentum and Impulse 








Participants 


18 


83 


94 


Non-Participants 


28 


82 


86 


Newton’s 2"'' Law 








Participants 


10 


70 


90 


Non-Participants 


39 


92 


97 



^ Includes those who rated the item a 4 or 5 on a five-point scale from 1 “not adequately prepared” to 5 
“very well prepared.” 



* By design, not all PTRAs participated in each workshop; the total number responding for each 
workshop to both the pre- and post-institute questionnaires is included in the table. 

* The change in participants’ perceptions of preparedness is statistically different than non-participants’ 
change (p < 0.05). 



When PTRAs did not feel well prepared to offer a workshop after participating in it during the 
institute, the post-institute questionnaire asked them to explain why the session did not better 
prepare them. The most common response, given by 19 of the 36 PTRAs replying to this 
question, was that they needed more time to become familiar with the workshop activities. Of 
these 19, 12 specifically mentioned needing more experience with the technology used in the 
activities. The second most common reason for not feeling prepared, given by 12 PTRAs, was 
that they did not agree with the workshop structure or pedagogy. Two PTRAs wrote: 
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[Workshop title] needs time to sink in and work through. It was well done, 1 just need 
time to allow the content to sink in. 



The [title] workshop was a little too scattered and did not present a cohesive program to 
present to other teachers. I would have liked more time with the [workshop title] 
stuff. ..so that I would feel more comfortable with [ it]. 

In an open-ended item on the post-institute questionnaire, PTRAs were asked to describe the 
single greatest impact the institute had on them. Twenty-six of the 53 respondents indicated an 
impact related to their classroom practice; 17 PTRAs specifically mentioned receiving activities 
to use in the classroom and 9 mentioned learning strategies for teaching students. As one PTRA 
wrote: 



1 have learned a lot that 1 can use to help my students, such as a new way to cover ] topic] 
and some lessons I haven ’t used before. 

In contrast, only 1 1 of the 53 responding PTRAs described an impact related to their abilities as a 
workshop leader. Of these, 8 indicated they gained new classroom activities to share in their 
outreach workshops. Only 3 mentioned learning strategies for working with adults. Similarly, 
the 10 interviewed PTRAs where asked what they gained from the summer institute. Four 
described learning teaching ideas to use in their classroom; three PTRAs mentioned learning 
workshop facilitation strategies. Given the heavy focus of the institute on classroom activities, it 
is not surprising that the PTRAs indicated that the greatest impact of the institute was improving 
their abilities as a classroom teacher. 

Still, even PTRAs who felt that the greatest impact was on their abilities as classroom teachers 
may have been impacted in other areas. Thus, HRI asked a number of questions specifically 
about the impacts of the institute on the PTRAs as professional development providers. The 
most common response to a question asking what they had learned that would help them in this 
role, given by 20 of the 47 responding PTRAs, was learning strategies for working with adults. 
However, 8 of these 20 indicated that the strategies for working with adults were modeled, but 
not discussed, which left it up to the individual PTRAs to determine what the key aspects of the 
strategy were. As one PTRA wrote: 

The presenters modeled the techniques of presenting effective workshops, but we did not 
really go into the theory. 

Other impacts included familiarizing the PTRAs with the AAPT/PTRA manuals and exposing 
them to the content to be addressed in the rural institutes (mentioned by 8 and 7 PTRAs, 
respectively). Seven PTRAs indicated that the institute had no impact on them as professional 
development providers, as it did not contain explicit discussions of professional development 
strategies, questioning techniques to use with adults, or the research on how people learn. 

The PTRAs were also asked a series of questions on the pre- and post-institute questionnaires 
regarding their attitudes and preparedness to lead professional development. These items were 
combined into three composite variables to reduce the unreliability associated with individual 
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survey items. (Definitions of the composite variables, a description of how they were created, 
and reliability information are included in Appendix A.) Each composite has a minimum 
possible score of 0 and a maximum possible score of 100. A score of 0 would indicate that a 
participant selected the lowest response option for each item in the composite, whereas a score of 
100 would indicate that a participant selected the highest response option for each item. 

The first composite measures PTRAs’ attitudes about the importance of a number of aspects of 
professional development, including developing teachers’ understanding of important physics 
content, and when and why to use an activity within their science curriculum. The second 
composite measures PTRAs’ feelings of preparedness to do these activities in their workshops. 
The third composite includes items relating to PTRAs’ preparedness to provide professional 
development in a variety of formats, such as leading a multi-day outreach institute focused on 
one or two core physics topics and planning workshop activities to meet the needs of outreach 
participants from a wide range of backgrounds. 

By linking PTRAs’ responses from the pre- and the post-institute questionnaires, HRI is able to 
examine changes in these composite scores. As can be seen in Table 7, these data indicate that 
even before the institute the PTRAs scored highly on these composites. The data also show that 
the PTRAs’ scores on these composites increased significantly after participating in the summer 
institute, another indication that the project is having an impact on the PTRAs. Effect sizes'* for 
these changes are 0.22 standard deviations for the attitudes composite, 0.71 standard deviations 
for the preparedness to provide professional development composite, and 0.75 standard 
deviations for the preparedness to provide professional development in a variety of formats 
composite. 



Table 7 

PTRAs’ Composite Scores 





Pre 


Post 




Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Attitudes about Professional Development 


90.33 


9.12 


92.40* 


7.92 


Preparedness to Provide Professional Development 


78.51 


16.31 


87.36* 


13.47 


Preparedness to Provide Professional Development in a Variety 
of Formats 


72.08 


17.22 


82.26* 


12.30 



* Post-questionnaire score significantly greater than pre-questionnaire score, p < 0.05. 



At the same time, from 4 to 21 percent of the PTRAs scored below 75 percent on these 
composites, and interviews with PTRAs and observations of PTRAs facilitating their rural 
institutes also provide evidence that some PTRAs do not feel well prepared to work with 
outreach participants. Observations of rural institutes parallel those of the PTRA institute. A 
majority of the time was spent having participants work through classroom activities as if they 
were students. As the PTRAs have reported that a substantial portion of outreach participants in 
the rural institutes do not have a strong grasp of the physics content, having teachers work 



“* Effect sizes of about 0.20 are typically considered small, 0.50 medium, and 0.80 large. Cohen, J. (1988). 
Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ; Lawrence Erlbaum Associates. 
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through the activities is an effective strategy for teaching them the content. HRI also observed 
PTRAs discussing logistical issues regarding classroom implementation of the activities, though 
this type of discussion occurred less frequently. However, HRI rarely saw PTRAs moving the 
discussion beyond physics content or the logistics of implementing a classroom activity. 

While improving teachers’ content knowledge and raising their awareness of instructional 
resources is necessary for improving the teaching and learning of physics, it is probably not 
sufficient for having the magnitude of impacts to which the project aspires. Understanding what 
ideas (both correct and incorrect) students are likely to have prior to instruction and the 
implications of those ideas for teaching and learning are examples of the type of knowledge 
(often termed “pedagogical content knowledge”) teachers need to be effective. Given that the 
PTRAs already have a great deal of experience sharing classroom activities with teachers, the 
project may want to increase the emphasis on pedagogical content knowledge throughout the 
summer institute and decrease the emphasis on classroom activities. In order to be successful, 
the PTRAs will need opportunities to practice and receive feedback on incorporating these types 
of activities and discussions into their workshops. 

Data collected on the post- institute questionnaire support both the need, and the PTRAs’ desire, 
for the summer institute to include a greater focus on pedagogical content knowledge and on 
effective professional development. The PTRAs were asked again the extent to which they 
needed leadership training in a number of areas. It was originally hypothesized that participation 
in the PTRA summer institute would decrease PTRAs’ need for additional training. As can be 
seen in Table 8, even after attending the institute, many PTRAs indicated they would benefit 
from additional training in areas that would help them as a professional development leader (i.e., 
effective professional development strategies, research on misconceptions/student thinking in 
physics). The lack of a decrease in many of these areas, and a significant increase in the need for 
additional training in the research on common misconceptions/student thinking in physics, may 
be an indication that the PTRAs are beginning to realize that being an effective professional 
development provider requires them to do more than share classroom activities with teachers. 



Table 8 



PTRAs Indicating a Substantial Need^ for Additional Leadership Training 





Percent of Respondents 
(N = 52) 


Pre' 


Post 


Difference 


The research on common misconceptions/student thinking in physics 


22 


48 


26* 


The research on the principles of effective professional development 


32 


48 


14 


Strategies for implementing the principles of effective professional development 








in workshops for other teachers 


35 


48 


13 


The research on how people learn 


37 


42 


5 


Technologies for physics instruction 


24 


24 


0 


Activities for physics instruction 


12 


8 


-4 


Physics content 


13 


8 


-5 



Includes those who rated the item 4 or 5 on a five-point scale from 1 “I don’t need any additional training” to 5 “I could use a 
lot of additional training.” 

* The pre-institute percents shown here are different than those shown in Table 2 as this table includes responses only from 
those PTRAs that completed both a pre- and post-institute questionnaire. 

* Indicates a significant change in PTRAs’ perceived need for additional training (p < 0.05). 
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These data are supported by PTRAs’ responses to an open-ended item on the post-institute 
questionnaire that asked what additional training would be helpful in preparing them for their 
role as a professional development provider. The most common request was for more training 
on how to facilitate professional development, given by 17 of the 41 PTRAs responding to this 
question. Ten of the respondents requested more information on how students learn 
physics/common misconceptions, 7 indicated they wanted more information on physics 
education research in general, and 6 asked for specific information on running a rural institute 
(e.g., timelines). Examples of comments made by PTRAs include: 

I need more help to learn how to help other teachers become reflective practitioners 
without seeming to be arrogant. I want to help people think about student 
misconceptions and how to ask probing open-ended questions to students. 

I would like to learn more about the principles of effective professional development and 
what strategies are available to use to implement them in workshops for other teachers. 

[I want] more practice with modeling. More about adult learners, since they’re very 
different from high school students. 

I would like to have a better understanding of what research tells about professional 
development [and] how to implement these findings. 

On both the post- institute questionnaire and during interviews, PTRAs were asked what aspects 
of the institute could have been better. Other than issues with the institute facilities (i.e., dorms, 
meals, etc.), the most common responses, mentioned by 10 of the 40 respondents on the post- 
institute questionnaire and 4 of the 10 interviewees, was a request for more information on 
facilitating outreach workshops and on the educational research on student thinking in physics. 
The PTRAs asking for these improvements in their training often made suggestions as to how to 
incorporate them into the institute. For example: 

Bring a group of rural teachers from areas we don’t serve and have us spend a day each 
helping them and helping us show we can do a workshop. 

Bring in some physics education research folks and let us get information from leaders in 
the field. 

We could do much more with talking about physics education research and what we are 
hoping the teachers take back from our workshops and how that interacts with what 
students are thinking... we should look at the manuals we already have and look about 
how to incorporate that [into the manuals]. 



Implications 

Looking across the data on the summer institute and its impacts on the PTRAs provides some 
valuable insights into the preparation of the PTRAs for their role as professional development 
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providers. First, the PTRAs are primarily approaching the summer institute with the mindset of 
a classroom teacher, looking for activities and teaching strategies to use in their classrooms. The 
data also show that the summer institute was well-aligned with this mindset; large portions of the 
workshops were spent working through the student activities in the AAPT/PTRA manuals. 
Predicatively, the PTRAs reported that the greatest impact of the institute was their gaining 
activities and strategies to use in their classroom teaching. 

Another theme that emerged from the data is that the PTRAs have a desire to incorporate more 
of the physics education research and what is known about best practices in professional 
development into their outreach workshops. Given the vital role the institute workshops play in 
preparing the PTRAs, the project leadership may need to expend a greater effort ensuring that the 
summer institute workshops incorporate opportunities for the PTRAs to learn and hone these 
types of skills. PTRAs, like most people, tend to teach as they were taught. If the summer 
institute workshops focus on working through classroom activities, it is not surprising that the 
PTRAs’ outreach workshops have the same focus. 

There are a number of approaches the leadership could take to increase the emphasis on 
leadership in the summer institute workshops. One possibility would be to ask the summer 
workshop leaders to annotate each classroom activity with the relevant research on student 
misconceptions, a description of how the activity addresses those misconceptions, and an 
explanation of how the activity contributes to the conceptual development of important physics 
ideas. Another possibility is to have the summer workshop leaders reduce the number of 
activities they work through with the PTRAs and devote more time to developing the PTRAs’ 
professional development provider skills. For example, the leaders might want to have the 
PTRAs examine samples of student work and practice leading discussions around them. The 
leaders could then, along with the other participants, provide constructive feedback to the PTRAs 
on their leadership skills. Given that the PTRAs have requested a greater emphasis on the 
physics education research and what is known about effective professional development, it is 
likely that a well implemented shift in the emphasis of the institute would be welcomed by the 
PTRAs. 



2004 Rural Institutes 

As noted earlier, the main goals of the AAPT/PTRA Rural project focus on improving the 
teaching and learning of physics/physical science in rural classrooms via the rural regional 
centers. The project’s model is for each center to host a 30-hour summer institute, and two, six- 
hour follow-up sessions during the school year. The summer institute is intended to focus on a 
small number of physics topics and provide outreach participants the opportunity for in-depth 
study of both the physics content and teaching strategies. The two follow-up sessions are 
intended to give outreach participants an opportunity to revisit the topic and reflect upon their 
attempts to incorporate what they learned into their classroom teaching. 

Each rural regional center operates in conjunction with a local university and has a designated 
Rural Regional Coordinator, typically a member of the university’s physics department. The 
coordinator’s responsibilities include recruiting outreach participants, arranging facilities and 
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equipment for the institutes, and managing all of the necessary paperwork. The coordinator 
makes it possible for the PTRAs to focus their energies on designing and implementing the 
professional development. 

This section of the report describes the quality and impacts of the 2004 rural institutes. Data 
come from project records of participant attendance, a questionnaire administered to all rural 
institute participants, HRTs observations of portions of two rural institutes, pre-institute 
questionnaire responses from PTRAs who led rural institutes, and focus group interviews with a 
sample of Lead PTRAs. 



Participation in the Rural Institutes 

The AAPT/PTRA Rural project operated 25 rural regional centers during its third year. Three of 
the centers were continuations of “prototype” institutes created to test the logistics of this model 
prior to NSF funding, one center was initiated in the project’s first year, seven in the second year, 
and 14 in the third year of NSF funding. Table 9 shows the number of outreach participants 
attending each of the rural institute summer and follow-up sessions.^ Overall, 521 teachers 
attended the rural institutes; 59 percent attended a follow-up session during the school year. 
Fifty-six percent of the outreach participants reached the goal of 36 hours of professional 
development during a year. 

The lower attendance at the follow-up workshops can be explained, in part, by the difficulty of 
finding a meeting time that works for all participants in a rural center. Another issue is that some 
participants are located hundreds of miles from the rural center, making travel for a one-day 
workshop too difficult. If the project believes the follow-up workshops are an important part of 
the professional development, it will need to seek ways to increase attendance at these sessions. 
Given that some centers have higher rates of participation in the follow-up sessions, the project 
may want to create a mechanism for center leaders to share strategies they have found effective 
for boosting attendance. 



^ Participation data come from AAPT/PTRA Rural project records and are current as of March 28, 2005. 
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Table 9 



Outreach Participants Attending each Session, by Rural Regional Center 





Number of Outreach Participants 


Summer 

Rural 

Institute 


Any 

Follow-Up 

Session* 


At Least 36 
Hours of 
Professional 
Development 


Brigham Young University 


14 


14 


14 


Central Pennsylvania 


22 


13 


13 


Coastal Carolina University^ 


24 


0 


0 


Colby College 


20 


17 


17 


Colgate University 


25 


20 


20 


Colorado School of Mines 


7 


6 


6 


Eastern Kentucky University 


20 


7 


5 


Emporia State University 


22 


17 


17 


Erostburg State University 


19 


0 


0 


Georgia College and State University 


22 


18 


18 


Gonzaga University 


15 


0 


0 


Idaho State University 


43 


36 


36 


Illinois State University^ 


16 


0 


0 


James Madison University 


20 


17 


16 


Lee College 


15 


16 


14 


Montana State University 


21 


13 


13 


Ohio State University 


18 


15 


13 


Saginaw Valley State University 


25 


21 


19 


South Dakota State University' 


24 


10 


9 


State University of New York - Eredonia 


17 


13 


13 


Texas A&M University 


28 


0 


0 


Texas Tech University 


36 


29 


26 


University of Pittsburg - Bradford 


20 


11 


10 


University of Wisconsin - River Palls 


14 


8 


8 


Youngstown State University 


14 


7 


6 


Total 


521 


308 


293 



“Prototype” center 

* The length of follow-up workshops varied between two-hour sessions on single days and 12-hour sessions on two 
days. 



In addition to the goal of providing at least 36 hours of professional development to participants 
per year, the project has the larger goal of providing at least 108 hours of professional 
development over the course of three years. By combining participant data from the past several 
years, it is possible to examine the project’s progress towards reaching this goal. As can be seen 
in Table 10, only 23 of the 162 outreach participants in centers that have existed for three years 
have reached the goal of 108 hours of professional development. However, it is encouraging that 
the third cohort of sites (those that began in 2003) has been much more successful at retaining 
participants for a second year. 
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Table 10 



Retention Rates for Rural Regional Centers, by Inaugural Year of the Center 





Number of Outreach Participants 


Inaugural Year of Rural Regional Center 


2001^ 


2002 


2003 


2004 


1 Rural Institute Attended 


68 


25 


58 


271 


2 Rural Institutes Attended 


35 


11 


136 


— 


3 Rural Institutes Attended 


16 


7 


■ ■ 


— 


Total 


119 


43 


194 


271 



’ “Prototype” centers 



The Outreach Participants 

An outreach participant questionnaire administered at the beginning of each rural institute 
collected a variety of information on the outreach participants. Since the questionnaires were 
administered on-site at the beginning of each institute, a 100 percent response rate was achieved. 
As can be seen in Table 11, half of the outreach participants were female and nearly all were 
white. Eighty-three percent taught high school during the 2003-04 academic year. Eighty-five 
percent of the participants taught physics and/or physical science; 64 percent taught physics, and 
over 55 percent taught physical science. Given the project’s target audience of rural teachers, it 
is not unexpected that 70 percent of the outreach participants taught other science subjects and 
more than 1 in 4 taught non-science classes. 

Table 12 shows the number of semesters of college coursework completed by the outreach 
participants. Eorty-three percent of the outreach participants have taken eight or more college 
semesters of physics/physical science while 30 percent have taken three or fewer semesters. 
These data indicate that there is a broad range of content backgrounds among participants. 
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Table 11 



Demographic Data for Outreach Participants 





Percent of Participants 


Gender 




Male 


49 


Female 


51 


Race 




White 


94 


Black or African-American 


3 


American Indian or Alaskan Native 


1 


Asian 


1 


Hispanic or Latino 


1 


Native Hawaiian or Other Pacific Islander 


0 


Grade Level Taught^ 




High School 


83 


Middle School 


25 


Elementary School 


3 


Not a Classroom Teacher 


5 


Prior Teaching Experience 




0-2 Years 


15 


3-5 Years 


16 


6-10 Years 


22 


11-20 Years 


28 


21 or More Years 


19 


Teaching Assignment Includes^ 




Physics 


64 


Physical Science 


55 


Other Science 


70 


Non-Science 


28 



Percents may add to more than 100 as participants could select more than one category. 



Table 12 



Outreach Participants’ College Course work 





Percent of Participants 


0 

Semesters 


1-3 

Semesters 


4-7 

Semesters 


8 or More 
Semesters 


Life Science/Biology 


10 


30 


14 


45 


Chemistry 


8 


25 


23 


45 


Physics/Physical Science 


5 


25 


26 


43 


Mathematics 


1 


28 


30 


40 


Earth/Space Science 


22 


42 


18 


19 


Engineering/T echnology 


39 


35 


12 


14 



The Quality and Impacts of the Rural Institutes 

Most of the 2004 rural institutes took place prior to the 2004 PTRA institute. Thus, HRI was 
able to ask the PTRAs on the pre-institute questionnaire and during focus group interviews held 
during the summer institute about their work with rural outreach participants. On an open-ended 
questionnaire item asking about the goals of the rural institutes, 47 of the 66 respondents 
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indicated that a goal of their institute was to share classroom teaching strategies with 
participants; 34 mentioned working to deepen participants’ knowledge of physics concepts. 
These goals were also the two most common ones described during focus group interviews with 
Lead PTRAs. This PTRA’s comment is typical: 

Teachers participating in workshops I help lead should develop more confidence in their 
own understanding of important physics concepts and how to help their students learn 
them. 

HRI’s observations at two centers indicate that the institutes focused heavily on having the 
outreach participants work through student activities, with the goal of deepening their physics 
content knowledge. The goal of increasing outreach participants’ repertoire of teaching 
strategies appeared to occur to a lesser extent, and tended to consist of the outreach participants 
becoming familiar with the classroom activities in the AAPT/PTRA manuals. Although HRI 
observed some discussions of classroom strategies specific to teaching physics/physical science, 
the majority of the discussions focused on logistical issues related to the preparation and 
implementation of student activities. 

PTRAs’ comments on the successes from their rural institutes on the pre-institute questionnaire 
correlated with HRI’s observations. Increasing participants’ physics content knowledge and 
provided participants with classroom activities were each listed as successes of the rural 
institutes by 14 of the 28 PTRAs responding to this question. As these three PTRAs described: 

Teachers in grades K-12 got a broader understanding of physics content. They will be 
using [ the PTRA ] materials in their classrooms. 

The mixture of classroom activities, technology, and discussion of concepts seems to 
reach almost everyone in one way or another. 

We’ve opened a lot of eyes. So many of our participants needed the physics concepts 
along with activities. We even straightened out lots of their misconceptions. 

During the focus group interviews. Lead PTRAs reported a variety of strategies which they 
found to be effective in helping them reach their goals for the rural institutes. The strategies 
mentioned by the PTRAs were aligned with four goals: deepening outreach participants’ 
knowledge of content and pedagogy; formatively assessing participant understanding during the 
workshop; creating a supportive environment for participants; and improving the logistics and 
planning for the institute. 

One strategy the Lead PTRAs found effective for deepening outreach participants’ knowledge of 
content and pedagogy was holding explicit discussions about the pedagogy modeled by the 
PTRAs. Another was asking the outreach participants to wear different hats. For example, the 
PTRAs first had outreach participants consider an activity as a student, asking questions such as, 
“What did you find as far as looking at this from the students’ perspective?” The Lead PTRA 
indicated that this approach allowed the participants to feel safe in voicing their understanding 
(or lack thereof) of the content. After considering an activity from a student’s perspective, the 
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PTRAs asked the participants to think about the activity as a teacher, using questions such as, 
“Now, let’s put our teacher hat back on. . .What did you find was a hassle from the perspective of 
concepts or the methods you might use to do that?” 

Examples of strategies for formatively assessing outreach participants’ understanding included 
starting each day with a review of the previous day, a question/answer session, or using short 
tasks (e.g., Tom Okuma’s ranking tasks, problems based on physlets). One institute asked 
participants to keep a reflective journal in which they answered the following questions at the 
end of each day: “What did you learn today?” “What puzzled you?” and “Any comments?” The 
PTRAs then read and made non-judgmental comments in the journals, and used the participants’ 
responses to help tailor the workshop for the next day. 

Using cooperative games, having participants share teaching strategies, and working in teams to 
solve problems together outside of the workshop time (e.g., a take home project) were among the 
methods PTRAs shared for creating a supportive environment at their rural institutes. 

Suggestions Lead PTRAs had for improving the logistical side of a rural institute included 
involving multiple PTRAs in the planning and implementation of an institute, and grouping 
participants by level of need and working with the groups in different rooms. One institute used 
participants’ strengths and weaknesses in content, pedagogy, and technology to group 
participants; each group contained someone strong in each area. Another workshop found it 
effective to keep switching groups around during the week. 

PTRAs were also asked during focus group interviews and on the pre-institute questionnaire 
about any obstacles they may have encountered while working with teachers at their rural 
institutes. Most of the Lead PTRAs in the focus groups (9 of the 12) indicated that it is difficult 
to plan for and address the needs of teachers who teach at different grade levels and have varying 
levels of background knowledge/training in physics. This issue was also among the most 
common responses to this question on the pre-institute questionnaire, mentioned by 7 of the 30 
responding PTRAs. Another seven PTRAs mentioned participant turnover as an obstacle to 
planning for the institute as they could not build upon work from the previous summer nor did 
they know their participants’ strengths and weaknesses ahead of time. 

The PTRAs offered a variety of suggestions for improvements they would make to their outreach 
institutes. During the focus group interviews, the Lead PTRAs were asked what they would do 
differently the next time they led a rural institute. Their responses fell into two categories: 
changes in logistics, and changes in the way they implement their workshops. Suggestions for 
logistical changes included having the same PTRAs year after year as they know the skill level 
and needs of the outreach participants; involving the Rural Regional Coordinator in the institute 
planning and implementation process; and collecting a refundable registration fee from outreach 
participants to reduce dropout. 

The PTRAs also discussed changes they would like to make to their workshop content and 
pedagogy. These included supplementing the PTRA manuals with outside materials; making the 
pedagogy they use during the rural institute more explicit to the participants; and incorporating 
formative assessment activities to check whether participants are “getting it.” One PTRA 
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mentioned a desire to build in more connections between the daily activities and the overall 
storyline of the week’s focus. 

While the rural institutes, like all other teacher enhancement programs, can continue to improve, 
there is evidence that they are having an impact on participating teachers. The Outreach 
Participant Questionnaire asked the outreach participants about their opinions about science 
teaching, perceptions of content and pedagogical preparedness, and frequency of use of various 
teaching practices. These items were combined into six composite variables (see Appendix A). 

By linking data across years, HRI is able to examine changes in these composite scores for 
participants completing the questionnaire on multiple occasions. Table 13 shows composite 
scores for participants with two data points.® The data indicate a significant increase in 
participants’ perceptions of pedagogical preparedness across the two time points (an effect size 
of 0.24 standard deviations). Participants also report a significant increase in their frequency of 
use of investigative teaching practices and a significant decrease in their frequency of use of 
traditional teaching practices (effect sizes of 0.30 and -0.24 standard deviations, respectively). 
As the number of rural centers increases, the number of participants submitting questionnaires 
across the targeted, three-year time frame should increase considerably, allowing for a broader 
look at the impact of the project. 



Table 13 



Outreach Participants’ Composite Scores 





N 


Initial Year 


Most Recent Year 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Attitudes Toward Standards-Based Teaching 


159 


77.38 


12.76 


76.30 


12.56 


Pedagogical Preparedness* 


151 


52.66 


17.09 


55.91 


15.95 


Physics Preparedness 


163 


53.12 


20.69 


55.17 


19.84 


Traditional Teaching Practices* 


91 


67.99 


15.21 


64.81 


14.09 


Investigative Teaching Practices* 


91 


31.52 


13.46 


34.76 


12.54 


Investigative Classroom Culture 


95 


65.13 


16.01 


66.43 


15.75 



* Initial Year score significantly different than Most Recent Year score, p < 0.05. 



Although the outreach participants’ perceptions of their content preparedness have not changed 
significantly, results from a teacher content knowledge impact study indicate that the PTRA 
program is having a positive impact on participants’ physics content knowledge^ The study 
took advantage of the different cohorts of rural institutes, using participants from the momentum 
and energy institutes as a control group for the kinematics and dynamics institutes (and vice 
versa). The assessment utilized contained four scales aligned with the topics covered in the 



® Data for the teaching practice and investigative classroom culture composites are presented only for those 
participants whose teaching assignment did not change from one year to the next. 

^ For a full description of the study, see Banilower, E.R., & Fulp, S.L. (2005). Results of the 2004 AAPT/PTRA 
Rural Institute Teacher Impact Study. Chapel Hill, NC: Horizon Research, Inc. 



Horizon Research, Inc. 



22 



May 2005 









institutes: kinematics, dynamics, momentum, and energy. In addition, the study examined 
whether changes in teacher test scores vary by teacher gender and grade-level taught. 



The study found that the AAPT/PTRA rural institutes have had a positive impact on teachers’ 
physics content knowledge. On each of the four test scales (kinematics, dynamics, momentum, 
and energy), controlling for pre-test scores and demographics, participants who had taken part in 
an institute on that topic scored significantly higher than participants who had not. The study 
also found that gender was not a significant factor in teacher learning, with the exception of 
dynamics, where males scored slightly higher on the post-test (controlling for pre-tests score) 
than females. Grade level taught was a significant factor on both the momentum and energy 
scales; even after controlling for pre-test scores, high school teachers scored higher on the post- 
test than elementary and middle school teachers. No differences were found for grade level 
taught on the kinematics or dynamics scales. 



Implications 

Two main themes emerge from the rural institute data. The first theme centers on retaining 
outreach participants. Of the 521 teachers who attended the rural institutes this year, 59 percent 
attended a follow-up session during the school year, and just 56 percent reached the goal of 36 
hours of professional development during a year. In addition, only 23 of the 162 outreach 
participants in centers that have existed for three years have reached the goal of 108 hours of 
professional development. On a positive note, the third cohort of sites (those that began in 2003) 
has been much more successful than earlier cohorts at retaining participants for a second year. 
The project may want to foster opportunities for the Rural Regional Coordinators and Lead 
PTRAs to share ideas on these issues, perhaps by creating mechanisms for them to share 
strategies that has worked well. 

The second theme deals with tailoring the rural institutes to the needs of the local teachers. The 
project is providing professional development to teachers with a wide range of backgrounds and 
needs. Nearly all of the interviewed and surveyed PTRAs stressed the difficulty of planning for 
and addressing the needs of teachers who teach at different grade levels and have varying levels 
of background knowledge/training in physics. The project may want to explore ways of 
gathering information from the rural institute participants, perhaps on the rural institute 
application form, regarding their comfort level with physics topics and the level and types of 
technology available to them at their schools. Given the commonality of this issue across sites, 
the project may also want to consider building opportunities into the PTRA institute for 
discussions around workshop planning and implementation strategies that have proven effective 
in addressing the diverse needs of the rural teachers. 

Third, in order to maximize the project’s impact on physics/physical science teaching and 
learning, the project may need to increase the focus during the rural institutes on developing 
outreach participants’ pedagogical content knowledge. For example, it may be beneficial to 
include a greater focus on developing participants’ questioning skills, knowledge of common 
student misconceptions, and ability to informally assess student understanding. Lead PTRAs in 
the focus groups commented on a desire to make the pedagogy they use during the rural institute 
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more explicit to the participants and integrate formative assessment activities throughout their 
workshops. To increase the likelihood of success, it will be important to provide the PTRAs 
with opportunities to develop and practice the skills necessary to include these types of activities 
in the rural institutes. 



Student Impact Study 

In the summer of 2004, HRI and the AAPT/PTRA rural project leadership developed a study for 
investigating the impact of the AAPT/PTRA rural project on student achievement. The study 
focuses on student achievement in kinematics, dynamics, momentum, and energy. With the 
help of the project, HRI recruited 172 teachers to participate in the study. A random sample of 
89 teachers, stratified by rural institute attended, was selected from the volunteers; 45 of the 
teachers participated in a kinematics and dynamics institute, and 44 participated in a momentum 
and energy institute. 

The study utilizes a pre-test/post-test design, with teachers from the kinematics and dynamics 
institute serving as a comparison group for teachers from the momentum and energy institute 
(and vice versa). The 50-item assessment utilized in this study was developed jointly by HRI 
and the AAPT/PTRA leadership, and included some items from previously developed 
assessments such as Jim Minstrell’s Diagnoser and Ron Thornton and David Sokoloff s Force- 
Motion Concept Evaluation. 

Because HRI has not yet received data from some of the participating teachers, results of this 
study are not available for this report. HRI will provide the project a supplemental report when 
the results of the study are available. 



Summary and Recommendations 

The AAPT/PTRA Rural project has made much progress in its third year. The project 
successfully established an additional 14 Rural Regional Centers, and provided professional 
development to 521 teachers across 25 sites. The professional development appears to be having 
a positive impact on outreach participants’ perceptions of pedagogical preparedness and their 
frequency of use of investigative teaching practices. Data from the teacher impact study also 
indicate that the project has had a positive impact on participants’ physics content knowledge. 

The project has also had an impact on the preparedness of the PTRAs, both as classroom 
teachers, and to a lesser extent, as professional development providers. PTRAs reported gaining 
activities and instructional strategies to use in the classroom both. PTRAs also reported a 
positive impact on their attitudes toward and preparedness to implement professional 
development. 

Still, as is true with all teacher enhancement projects, there are areas in which the project has 
room to grow. The focus on classroom activities, including sharing activities and discussing the 
logistical implementation of the activities, appears to outweigh other goals at both levels of the 
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project: the PTRA institute and the rural institutes. Although the PTRAs are becoming adept at 
using the classroom activities to teach outreach participants core physics topics, other aspects of 
science teaching critical to effective practice have not yet been fully integrated into PTRA- 
provided professional development. Given the project’s target audience of under-prepared or 
cross-over physics/physical science teachers, making sure the PTRAs have the necessary 
knowledge and skills to address the content and pedagogical content knowledge needs of 
outreach participants is vitally important. In the spirit of a critical friend, HRI offers the 
following recommendations to the project. 

> The project should consider ways to increase the focus at the PTRA Summer Institute 
on working with adults in a professional development setting and decrease the focus on 
working with children in the classroom. This shift in focus needs to happen in both the 
training offered to the PTRAs hy the project and in the PTRAs’ understanding of their 
role at the Summer Institute. 

The data show that the PTRAs are entering the institute with a focus on what they can apply 
to their practice as a classroom teacher. Further, the PTRAs reported that the greatest impact 
of the institute was on their classroom practice. However, the primary purpose of the 
summer institute is to prepare the PTRAs for their role as professional development 
providers in the outreach institutes. Given that the PTRAs expressed a desire both before and 
after the institute for more training in facilitating professional development, effective 
professional development strategies and physics education research, it would appear that the 
PTRAs still have the need and interest to grow in these areas. 

Just like students need opportunities to practice and receive feedback on the skills they are 
expected to master, the PTRAs need opportunities to practice and receive feedback on their 
professional development facilitation skills. Providing the PTRAs with these opportunities 
may require decreasing the focus on familiarizing PTRAs with classroom activities during 
the PTRA institute and increasing the focus on practicing and receiving feedback on 
leadership skills (e.g., leading discussions, informally assessing participants’ understanding, 
helping outreach participants examine student work/focus on questioning strategies). These 
changes may require the project leadership to increase their work with the PTRA institute 
workshop leaders, helping them incorporate what is known from physics education research 
and the research on effective professional development. 

> The project should consider ways to increase the sharing of effective leadership 
strategies among PTRAs and Rural Regional Coordinators. 

Both the PTRAs and the Rural Regional Coordinators bring a wealth of experience and 
knowledge about what has been successful, and not so successful, in the planning and 
implementation of outreach institutes. This knowledge covers a range of topics from 
leadership skills (e.g., how to help outreach participants improve their questioning 
strategies), to increasing the retention rates of the rural institute, to strategies for dealing with 
diverse audiences (e.g., high school and middle school teachers, those with a degree in 
physics and those who have never taken a physics course). Pulling that knowledge together 
into a format that is both accessible and useful to other institute leaders and planners will 
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benefit not only the AAPT/PTRA Rural project but the greater professional development 
community as well. 

The project may want to consider what mechanisms already exist for such sharing (e.g., the 
PTRA listserv), and what other mechanisms may be needed. Some additional mechanisms 
may include having Rural Regional Coordinators attend the PTRA institute, reformatting the 
sharing session at the PTRA institute to focus on the sharing of professional development 
strategies, and asking Lead PTRAs to videotape portions of their institutes to view and 
discuss at the PTRA institute. Once the mechanisms are in place, the project will need to 
consider ways to ensure they are used effectively. For example, the project may want to 
“seed the discussion” on the listserv by asking the PTRAs and Rural Regional Coordinators 
to respond to specific questions or scenarios. Similarly, the project may want to first 
videotape the institutes run by members of the leadership team and lead a critical, but 
constructive, discussion about the quality of the professional development. Increasing the 
use of these types of mechanisms will also help the PTRAs keep their focus on their roles as 
professional development providers rather than their roles as classroom teachers. 
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Appendix A 

Analysis and Reporting of Questionnaire Data 



To facilitate the reporting of large amounts of survey data, and because individual questionnaire 
items are potentially unreliable, groups of survey questions that measure similar ideas can be 
combined into “composites.” Each composite represents an important construct related to 
science teaching or professional development. Cronbach’s Coefficient Alpha is a measure of the 
reliability of a composite (i.e., the extent to which the items appear to be measuring the same 
construct). A Cronbach’s Alpha of 0.6 is considered acceptable, 0.7 fair, 0.8 good, and 0.9 
excellent. 

Each composite is calculated by summing the responses to the items associated with that 
composite and then dividing by the total points possible. In order for the composites to be on a 
100-point scale, the lowest response option on each scale was set to 0 and the others were 
adjusted accordingly; so for instance, an item with a scale ranging from 1 to 5 was re-coded to 
have a scale of 0 to 4. As a result, someone who marks the lowest point on every item in a 
composite receives a composite score of 0 rather than some positive number. It also assures that 
50 is the true mid-point. The denominator for each composite is determined by computing the 
maximum possible sum of responses for a series of items and dividing by 100; e.g., a nine-item 
composite where each item is on a scale of 0-4 would have a denominator of 0.36. 



PTRA Pre- and Post-Institute Questionnaire Composite Definitions 



Table A-1 

Attitudes about Professional Development 



Questionnaire Composite 


Pre 


Post 


Develop their own understanding of important physics concepts 


Qllai 


Q8ai 


Understand student thinking and/or common misconceptions related to important physics concepts 


Qllbi 


Q8bi 


Examine science pedagogy /teaching strategies (e.g., white boarding, pair share) and when/why to 
use them 


Qllci 


Q8ci 


Understand when and why to use a particular activity within their science curriculum 


Qlldi 


Q8di 


Learn how to examine student work in order to assess student thinking and reflect on classroom 
practice 


Qllei 


Q8ei 


Identify/develop lessons aligned to learning goals and state and national standards 


Qllfi 


Q8fi 


Develop effective questioning strategies to elicit student understanding 


Qllgi 


Q8gi 


Informally assess student learning 


Qllhi 


Q8hi 


Formally assess student learning 


Qllii 


Q8ii 








Number of Items in Construct 


9 


9 


Reliability (Cronbacb’s Coefficient Alpha) 


0.83 


0.79 
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Table A-2 

Preparedness to Provide Professional Development 



Questionnaire Composite 


Pre 


Post 


Develop their own understanding of important physics concepts 


Qllap 


Q8ap 


Understand student thinking and/or common misconceptions related to important physics concepts 


Qllbp 


Q8bp 


Examine science pedagogy /teaching strategies (e.g., white boarding, pair share) and when/why to 
use them 


Qllcp 


Q8cp 


Understand when and why to use a particular activity within their science curriculum 


Qlldp 


Q8dp 


Learn how to examine student work in order to assess student thinking and reflect on classroom 
practice 


Qllep 


Q8ep 


Identify/develop lessons aligned to learning goals and state and national standards 


Qllfp 


Q8fp 


Develop effective questioning strategies to elicit student understanding 


QllSP 


Q8gp 


Informally assess student learning 


Qllhp 


Q8hp 


Formally assess student learning 


Qllip 


Q8ip 








Number of Items in Construct 


9 


9 


Reliability (Cronbacb’s Coefficient Alpha) 


0.94 


0.87 



Table A-3 

Preparedness to Provide Professional Development in a Variety of Formats 



Questionnaire Composite 


Pre 


Post 


Lead a six-hour AAPT/PTRA outreach workshop 


Q12a 


Q7a 


Lead a two- to five-day AAPT/PTRA outreach institute focusing on one or two core physics 
topics (e.g., kinematics and dynamics or energy and momentum) 


Q12b 


Q7b 


Conduct a demonstration lesson in an outreach participant’s classroom 


Q12c 


Q7c 


Coach an outreach participant (i.e., observe and provide feedback on a lesson) 


Q12d 


Q7d 


Provide on-going support to outreach participants via electronic media (e.g., email, listservs, on- 
line forums, etc.) 


Q12e 


Q7e 


Plan workshop activities that meet the needs of outreach participants with a wide range of 
backgrounds (e.g., middle school physical science and high school physics teachers) 


Q12f 


Q7f 








Number of Items in Construct 


6 


6 


Reliability (Cronbacb’s Coefficient Alpha) 


0.86 


0.83 
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Outreach Participant Questionnaire Composite Definitions 



Table A-4 

Attitudes Towards Based Teaching 



Questionnaire Composite 




Provide concrete experience before abstract concepts. 


Q8ai 


Develop students' conceptual understanding of science. 


Q8bi 


Make connections between science and other disciplines. 


Q8di 


Have students work in cooperative learning groups. 


Q8ei 


Have students participate in appropriate hands-on activities. 


Q8fi 


Engage students in inquiry-oriented activities. 


Q8gi 


Use computers. 


Q8ji 


Engage students in applications of science in a variety of contexts. 


Q8ki 


Use portfolios. 


Q8mi 


Use informal questioning to assess student understanding. 


Q8ni 






Number of Items in Construct 


10 


Reliability (Cronbacb’s Coefficient Alpha) 


0.75 



Table A-5 

Pedagogical Preparedness 



Questionnaire Composite 




Provide concrete experience before abstract concepts. 


Q8ap 


Develop students’ conceptual understanding of science. 


Q8bp 


Take students’ prior understanding into account when planning curriculum and instruction. 


Q8cp 


Make connections between science and other discipline. 


Q8dp 


Have students work in cooperative learning groups. 


Q8ep 


Have students participate in appropriate hands-on activities. 


Q8fp 


Engage students in inquiry-oriented activities. 


Q8gp 


Engage students in applications of science in a variety of contexts. 


Q8kp 


Use performance-based assessment. 


Q81p 


Use portfolios. 


Q8mp 


Use informal questioning to assess student understanding. 


Q8np 


Lead a class of students using investigative strategies. 


Q9a 


Manage a class of students engaged in hands-on/project-based work. 


Q9b 


Help students take responsibility for their own learning. 


Q9c 


Recognize and respond to student diversity. 


Q9d 


Encourage students' interest in science. 


Q9e 


Use strategies that specifically encourage participation of females and minorities in science. 


Q9f 


Involve parents in the science education of their students. 


Q9g 






Number of Items in Construct 


18 


Reliability (Cronbacb’s Coefficient Alpha) 


0.91 
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Table A-6 

Physics Content Preparedness (Version A) 



Questionnaire Composite 




Forces and motion 


QlOal 


Energy 


Q10a2 


Light and sound 


Q10a3 


Electricity and magnetism 


Q10a4 


Modern physics (e.g., special relativity) 


Q10a5 


Eormulating hypotheses, drawing conclusions, making generalizations 


QlObl 


Experimental design 


Q10b2 


Describing, graphing, and interpreting data 


Q10b3 






Number of Items in Construct 


8 


Reliability (Cronbach’s Coefficient Alpha) 


0.87 



Table A-7 

Physics Content Preparedness (Version B) 



Questionnaire Composite 




Kinematics (i.e.. Motion) 


QlOal 


Eorces (i.e., gravitational, normal, friction, tension) 


Q10a2 


Newton’s Laws 


Q10a3 


Linear Momentum 


Q10a4 


Energy (i.e.. Thermodynamics) 


Q10a5 


Energy as a Societal Issue 


Q10a6 


Static Electricity 


Q10a7 


Direct Current Circuits 


QlOaS 


Eormulating hypotheses, drawing conclusions, making generalizations 


QlObl 


Experimental design 


Q10b2 


Describing, graphing, and interpreting data 


Q10b3 






Number of Items in Construct 


II 


Reliability (Cronbach’s Coefficient Alpha) 


0.90 



Table A-8 

Traditional Teaching Practices 



Questionnaire Comosite 




Assign science/mathematics homework. 


Q13m 


Answer textbook/worksheet questions 


Q14g 


Review homework/worksheet assignments. 


Q14h 


Take short-answer tests (e.g., multiple choice, true/false, fill-in-the-blank). 


Q14y 






Number of Items in Construct 


4 


Reliability (Cronbach’s Coefficient Alpha) 


0.68 
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Table A-9 

Investigative Teaching Practices 



Questionnaire Composite 




Make formal presentations to the class. 


Q14d 


Engage in hands-on science activities. 


Q14k 


Design or implement their own investigation. 


Q14m 


Work on models or simulations. 


Q14o 


Work on extended science investigations or projects (a week or more in duration). 


Q14p 


Participate in field work. 


Q14q 


Write reflections in a notebook or journal. 


Q14s 


Work on portfolios. 


Q14x 






Number of Items in Construct 


8 


Reliability (Cronbach’s Coefficient Alpha) 


0.76 



Table A-10 

Investigative Classroom Culture 



Questionnaire Composite 




Arrange seating to facilitate student discussion. 


Q13d 


Use open-ended questions. 


Q13e 


Require students to supply evidence to support their claims. 


Q13f 


Encourage students to explain concepts to one another. 


Q13g 


Encourage students to consider alternative explanations. 


Q13h 


Participate in discussions with the teacher to further science understanding. 


Q14b 


Work in cooperative learning groups. 


Q14c 


Share ideas or solve problems with each other in small groups. 


Q14j 






Number of Items in Construct 


8 


Reliability (Cronbach’s Coefficient Alpha) 


0.80 
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Appendix B 



AAPT/PTRA: Pre-Institute Questionnaire 



AAPT/PTRA: Post-Institute Questionnaire 



Rural Institute Participant Questionnaire (Version A) 



Rural Institute Participant Questionnaire (Version B) 




AAPT/PTRA: Pre-Institute Questionnaire 2004 

Instructions: This questionnaire is for all PTRAs who will be attending the AAPT/PTRA Institute in Sacramento. Please complete 
this form using a #2 pencil or a blue or black ink pen. Darken circles completely. Erase or white out any stray marks. For all 
open-ended questions, please avoid writing in the markings at the side and top of the page; you can use the back of the page if 
necessary. All of your responses will remain strictly confidential in that they will be reported only as group results. 

Background and Expectations 

1. Was physics or physical science part of your teaching assignment last year? 

Yes No Did not teach last year (skip to question 3) 



2. Which best describes the location of the school where you teach? (Darken one circle.) 
Rural Urban Suburban 



3. How many years have you been teaching physics/physical science? Please enter this 
as a 2-digit number. For example, if 3 years, enter as 03; then fill in the 
corresponding circle in each column. 



4. In what year did you originally become a PTRA? (Darken one circle.) 

1985 1988 1994 1997 2000 2003 

1986 1992 1995 1998 2001 2004 

1987 1993 1996 1999 2002 



5. In which of the following organizations are you a member? (Darken all that apply.) AAPT NSTA 



6. Are you: (Darken one circle.) Male Female 



7. Which best describes you? 
(Darken one circle.) 



African-American (not of Hispanic origin) Hispanic 

American Indian or Alaskan Native White (not of Hispanic origin) 

Asian or Pacific Islander Other 



PLEASE DO NOT WRITE IN THIS AREA 
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8. What do you hope to gain from the upcoming PTRA Summer Institute? 



9. Only for PTRAs attending the summer institute for the first time'. Please describe how you learned about the PTRA program. 



Opinions and Preparedness 

Not Very 

adequately Somewhat well 

prepared prepared prepared 



c. Teaching about energy 

d. Teaching about societal issues related to energy 

e. Teaching about gravity 

f Teaching about kinematics 

g. Teaching about momentum and impulse 

h. Teaching about Newton’s 2^^ Law (i.e., dynamics) 

i. Incorporating graphical analysis into physics teaching 

j. Incorporating the use of PASCO computer interfacing devices into 
physics teaching 

k. Incorporating the use of Vernier computer interfacing devices into 
physics teaching 



10. How well prepared do you feel to conduct a workshop for other teachers in 
each of the following areas? (Darken one circle on each line.) 

a. Teaching about electricity (i.e., electrostatics, electric current, and 
circuits) 

b. Teaching about the electromagnetic spectrum 
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11. In the left section, please rate how important it is for teachers to do each of the following in a professional development 



program. In the right section, indicate how prepared you feel to help other teachers to 
each section on each line.) 

Importance 

Not Somewhat Very 

important imnortant imnortant 


do each one. (Darken one circle in 

Preparedness 

adequately Somewhat 

oreoared oreoared 


a. Develop their own understanding 
of important physics concepts 




b. Understand student thinking 
and/or common misconceptions 
related to important physics 
concepts 




c. Examine science teaching 
strategies (e.g., white boarding, 
pair share) and when/why to use 
them 




d. Understand when and why to use 
a particular activity within their 
science curriculum 




e. Learn how to examine student 
work in order to assess student 
thinking and reflect on classroom 
practice 




f Identify/develop lessons aligned 
to learning goals and state and 
national standards 




g. Develop effective questioning 
strategies to elicit student 
understanding 




h. Learn how to informally assess 
student learning 




i. Learn how to formally assess 
student learning 





12. How well prepared do you feel to do each of the following? (Darken one ^ ^ 

circle on each line.) adequately Somewhat 

prepared prepared 

a. Lead a six-hour AAPT/PTRA outreach workshop 

b. Lead a two- to five-day AAPT/PTRA outreach institute focusing on one or two 
core physics topics (e.g., kinematics and dynamics, or energy and momentum) 

c. Conduct a demonstration lesson in an outreach participant’s classroom 

d. Coach an outreach participant (i.e., observe and provide feedback on a lesson) 

e. Provide on-going support to outreach participants via electronic media (e.g., 
email, listservs, on-line forums, etc.) 

f Plan workshop activities that meet the needs of outreach participants with a wide 
range of backgrounds (e.g., middle school physical science and high school 
physics teachers) 

PLEASE DO NOT WRITE IN THIS AREA 
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well 

prepared 
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13. To what extent would additional training in each of the following 
areas help you to be a more effective professional development 
provider? (Darken one circle on each line.) 



I do not I could use a moderate a good I could use 
need any a little amount of deal of a lot of 

additional additional additional additional additional 

training training training training training 



I could use I could use 



a. Physics content 

b. Activities for physics instruction 

c. Technologies for physics instmction 

d. The research on common misconceptions/student thinking in physics 

e. The research on how people learn 

f The research on the principles of effective professional development 
g. Strategies for implementing the principles of effective professional 
development in workshops for other teachers 

14. In general, what are your goals for the AAPT/PTRA outreach workshops you lead (i.e., what do you hope teachers will gain by 
participating)? 



15. Only for PTRAs who have conducted rural outreach institute workshops: What successes have you had in your rural institute 



workshops? 



16. Only for PTRAs who have conducted rural outreach institute workshops'. What obstacles have you encountered as you worked 
with teachers in rural institutes? 



THANK YOU!! 

• • • • • 



PLEASE DO NOT WRITE IN THIS AREA 
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AAPT/PTRA: Post-Institute Questionnaire 2004 

Instructions: Please complete this form using a #2 pencil or a black or blue pen. Darken circles completely. Erase or white out any 
stray marks. For all open-ended questions, please avoid writing in the markings at the side and top of the page; you can use the back 
of the page if necessary. All of your responses will remain strictly confidential in that they will be reported only as group results. 



1 . Indicate whether/when you participated in each of the following 
sessions as a PTRA. (Darken all that apply on each line.) 



Have never Participated Participated 

participated in a previous year this year 




2. For the workshops you participated in this year, please explain why you selected them. 



3. What was the single greatest impact the summer institute had on you (i.e., how have you changed as a result of the summer 
institute)? 
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4. For the workshops you did participate in this year, please rate the quality of instruction in each. (Darken one circle on each 

line.) Did not 

participate 

this year Poor Excellent 

a. Electricity (i.e., CASTLE) 

b. The Electromagnetic Spectrum (i.e., NASA workshop) 

c. Energy 

d. Energy - Societal Issues 

e. Graphical Analysis 
f Gravity 

g. Interfacing - PASCO 

h. Interfacing - Vernier 

i. Kinematics 

j. Leadership and Workshop Methods 

k. Make and Take 

l. Momentum and Impulse 

m. Newton’s 2*^^ Law (i.e., dynamics) 



5. Indicate how well prepared you feel to present a workshop based on each of the 
following, regardless of whether you have participated in an AAPT/PTRA 
workshop on that topic. (Darken one circle on each line.) 

adequately Somewhat well 

prepared prepared prepared 

a. Teaching about electricity (i.e., electrostatics, electric current, and circuits) 

b. Teaching about the electromagnetic spectrum 

c. Teaching about energy 

d. Teaching about societal issues related to energy 

e. Teaching about gravity 

f Teaching about kinematics 

g. Teaching about momentum and impulse 

h. Teaching about Newton’s 2*^^ Law (i.e., dynamics) 

i. Incorporating graphical analysis into physics teaching 

j. Incorporating the use of PASCO computer interfacing devices into physics 
teaching 

k. Incorporating the use of Vernier computer interfacing devices into physics 
teaching 



6. For each of the workshops that you attended this year in Sacramento and feel relatively less well prepared to present, please 
note the workshop in the space below and explain why you think the session did not better prepare you. 
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Not 

adequately 

prepared 



Somewhat 

prepared 



Very 

well 

prepared 



7. How well prepared do you feel to do each of the following? (Darken one 
circle on each line.) 

a. Lead a six-hour AAPT/PTRA outreach workshop 

b. Lead a two- to five-day AAPT/PTRA outreach institute focusing on one or two 
core physics topics (e.g., kinematics and dynamics or energy and momentum) 




f Plan workshop activities that meet the needs of outreach participants with a wide 
range of backgrounds (e.g. middle school physical science and high school physics 
teachers) 



8. In the left section, please rate how important it is for teachers to do each of the following in a professional development program. 
In the right section, indicate how prepared you feel to help other teachers to do each one. (Darken one circle in each section on 



each line.) 



Importance 



Preparedness 



Very 



Not Somewhat Very adequately Somewhat well 

important important important prepared prepared prepared 

a. Develop their own understanding 
of important physics concepts 

b. Understand student thinking 



and/or common misconceptions 



related to important physics 
concepts 

c. Examine science 
pedagogy/teaching strategies (e.g., 
white boarding, pair share) and 
when/why to use them 

d. Understand when and why to use a 



particular activity within their 
science curriculum 

e. Learn how to examine student 
work in order to assess student 
thinking and reflect on classroom 
practice 



f Identify/develop lessons aligned to 
learning goals and state and 
national standards 

g. Develop effective questioning 
strategies to elicit student 
understanding 

h. Informally assess student learning 

i. Formally assess student learning 



PLEASE DO NOT WRITE IN THIS AREA 
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9 . 



To what extent did each of the following occur during the institute in 
Sacramento? (Darken one circle on each line.) 



To a 

great extent 



a. Learned physics content 

b. Gained activities for physics instruction 

c. Gained experience with technologies for physics instruction 

d. Learned about the research on common misconceptions/ student thinking in 
physics 

e. Learned about the research on how people learn 

f Learned about the research on the principles of effective professional 
development 

g. Learned strategies for implementing the principles of effective professional 
development into workshops for other teachers 



10. Responses to question 9 will give us quantitative data on the extent to which each of the listed items occurred during the 
summer institute, but we'd really like to know "the rest of the story." Please explain in your own words what you gained as a 
result of your participation in this summer’s AAPT/PTRA institute that will help you in your role as a professional 
development provider? 



11. To what extent would additional training in each of the following areas 
help you to be a more effective professional development provider? 
(Darken one circle on each line.) 



I do not 
need any 
additional 



I could use 
a little 
additional 



I could use 
a moderate 
amount of 
additional 



I could use 
a good 
deal of 
additional 



I could use 
a lot of 
additional 



training training training training training 
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12. Responses to question 1 1 will give us quantitative data on which of the listed areas PTRAs may need additional training, but 
we'd really like to know, in your own words, what additional training you would find helpful in your role as a professional 
development provider. Please use the space below to comment. 



13. What aspects of the entire institute in Sacramento were particularly good? This question refers only to the AAPT/PTRA 
institute, not the AAPT conference. 



14. What suggestions do you have for improving the AAPT/PTRA summer institute? 



15. Please use the space below for any other comments about the Rural AAPT/PTRA project. 



THANK YOU!! 

• • • • • 

PLEASE DO NOT WRITE IN THIS AREA 
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AAPT/PTRA 2004 Rural Institute Participant Survey (Version A) 

Instructions: Please use a #2 pencil or a blue or black pen to complete this questionnaire. Darken circles completely, but do not 
stray into adjacent circles. Be sure to erase completely or white out any stray marks. Please remove the label before you return the 
completed questionnaire to the workshop leader to ensure the anonymity of your responses. 

A, Teacher Demographic Information 

1. Are you: Male Female 

2. Race - Are you: (Darken one or more.) 

American Indian or Alaskan Native Flispanic or Latino 

Asian Native Hawaiian or Other Pacific Islander 

Black or African-American White 

3. For how many days did you participate in last year's AAPT/PTRA rural institute, including both the summer institute (up to five 
days) and school year follow-ups (up to two days)? 

0 1 2 3 4 5 6 7 

4. For each of the following subjects, please indicate (a) the number of semesters of college coursework you have completed, and 
(b) whether you are certified to teach it at the secondary level. (Darken one circle in each section on each line.) 




5. How many years have you taught prior to this school year? (Darken one circle.) 

0-2 3-5 6-10 11-15 16-20 21-25 26 or more 



6. Which of the following did you teach this past school year? (Darken each circle that applies.) 

Did not teach any K-12 science (skip to question 8) 

Elementary school science 
Middle school science 
High school science 

7. How many sections of each of the following courses did you teach this past school year? (Darken one circle on each line.) 

7 or 

0 1 2 3 4 5 6 more 

Physics (e.g.. Regular, Honors, Advanced Placement) 

Physical Science 
Other Science 
Other Non-Science 
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B, Teacher Opinions and Preparedness 



8 . 



In the left section, please rate each of the following in terms of its importance for effective science instruction in the 
grades you teach. In the right section, please indicate how prepared you feel to do each one. (Darken one circle in each 



section on each line.) Importance 

Not Somewhat Fairly Very 

important imnortant imnortant imnortant 


Preparation 

Not Fairly Very 

adequately Somewhat well well 

oreoared oreoared oreoared oreoar 


a. Provide concrete experience before abstract 
concepts. 




b. Develop students' conceptual understanding 
of science. 




c. Take students' prior understanding into 
account when planning curriculum and 
instruction. 




d. Make connections between science and other 
disciplines. 




e. Have students work in cooperative learning 
groups. 








f Have students participate in appropriate 
hands-on activities. 




g. Engage students in inquiry-oriented 
activities. 




h. Have students prepare 

proj ect/laboratory/research reports. 




i. Use calculators. 




j. Use computers. 








k. Engage students in applications of science in 
a variety of contexts. 




1. Use perfonnance-based assessment. 




m. Use portfolios. 




n. Use informal questioning to assess student 
understanding. 




0 . Use calculator/computer-based labs. 




p. Use graphing calculators. 





Please indicate how well prepared you feel to do each of 

Not Fairly Very 

the following. (Darken one circle on each line.) adequately Somewhat well well 

prepared prepared prepared prepared 

a. Lead a class of students using investigative strategies. 

b. Manage a class of students engaged in hands-on/project-based work. 

c. Help students take responsibility for their own learning. 

d. Recognize and respond to student diversity. 

e. Encourage students' interest in science. 

f Use strategies that specifically encourage participation of females and 
minorities in science. 

g. Involve parents in the science education of their students. 
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10. Within science, many teachers feel better prepared to teach some topics than others. How well prepared do you feel to teach 
each of the following topics at the grade levels you teach, whether or not they are currently included in your curriculum? 
(Darken one circle on each line.) 



Physics 

1 . Forces and motion 

2. Energy 

3. Light and sound 

4. Electricity and magnetism 

5. Modem physics (e.g., special relativity) 



Not Fairly Very 

adequately Somewhat well well 

prepared prepared prepared prepared 



b. Scientific methods and inquiry skills 

1. Formulating hypotheses, drawing conclusions, making generalizations 

2. Experimental design 

3. Describing, graphing, and interpreting data 



Questions 11-14 ask about your science teaching. Please answer for your first physics or physical science class of the day 
during this past school year. If you did not teach physics or physical science, please answer for your first science class of the 
day. If you did not teach any science, darken here and skip the remainder of this questionnaire. 

11. What was the subject of this class? Life science/Biology Physics 

(Darken one circle.) Earth/Space science Physical science 

Environmental science Integrated science 

Chemistry General science 

12. What grade level was it? (Darken one circle.) 

Elementary school science Middle school science High school science 



13. About how often did you do each of the following in your science 
instmction in this class? (Darken one circle on each line.) 



Never 



Rarely Sometimes Often All or 

(e.g., a few (e.g., once (e.g., once almost all 
times a or twice or twice science 
year) a month) a week) lessons 
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Sometimes Often 
(e.g., once (e.g., once 
or twice or twice 

a month) a week) 

a. Participate in student-led discussions. 

b. Participate in discussions with the teacher to further science 

understanding. 

c. Work in cooperative learning groups. 

d. Make formal presentations to the class. 

e. Read from a science textbook in class. 

f Read other (non-textbook) science-related materials in class. 

g. Answer textbook/worksheet questions. 

h. Review homework/worksheet assignments. 

i. Work on solving a real-world problem. 

j. Share ideas or solve problems with each other in small groups. 

k. Engage in hands-on science activities. 

l. Follow specific instructions in an activity or investigation. 

m. Design or implement their own investigation. 

n. Design objects within constraints (e.g., egg drop, toothpick 

bridge, aluminum boats). 

o. Work on models or simulations. 

p. Work on extended science investigations or projects (a week or 

more in duration). 

q. Participate in field work. 

r. Record, represent, and/or analyze data. 

s. Write reflections in a notebook or journal. 

t. Prepare written science reports. 

u. Use mathematics as a tool in problem-solving. 

V. Use calculators. 

w. Use computers. 

X. Work on portfolios. 

y. Take short-answer tests (e.g., multiple choice, tme/false, 

fill-in-the-blank). 

z. Take tests requiring open-ended responses (e.g., descriptions, 

explanations). 

aa. Engage in performance tasks for assessment purposes. 



14. About how often did students in this class take part in each of the 
following types of activities as part of their science instruction? 
(Darken one circle on each line.) 



Never 



Rarely 
(e.g., a few 
times a 
year) 



THANK YOU!! 

• • • • t 



AAPT/PTRA 2004 Rural Institute Participant Survey - Page 4 of 4 
PLEASE DO NOT WRITE IN THIS AREA 



All or 
almost all 
science 
lessons 



{FORMID} 



















AAPT/PTRA 2004 Rural Institute Participant Survey (Version B) 

Instructions: Please use a #2 pencil or a blue or black pen to complete this questionnaire. Darken circles completely, but do not 
stray into adjacent circles. Be sure to erase completely or white out any stray marks. Please remove the label before you return the 
completed questionnaire to the workshop leader to ensure the anonymity of your responses. 

A, Teacher Demographic Information 

1. Are you: Male Female 

2. Race - Are you: (Darken one or more.) 

American Indian or Alaskan Native Flispanic or Latino 

Asian Native Hawaiian or Other Pacific Islander 

Black or African-American White 

3. For how many days did you participate in last year's AAPT/PTRA rural institute, including both the summer institute (up to five 
days) and school year follow-ups (up to two days)? 

0 1 2 3 4 5 6 7 

4. For each of the following subjects, please indicate (a) the number of semesters of college coursework you have completed, and 
(b) whether you are certified to teach it at the secondary level. (Darken one circle in each section on each line.) 




5. How many years have you taught prior to this school year? (Darken one circle.) 

0-2 3-5 6-10 11-15 16-20 21-25 26 or more 



6. Which of the following did you teach this past school year? (Darken each circle that applies.) 

Did not teach any K-12 science (skip to question 8) 

Elementary school science 
Middle school science 
High school science 

7. How many sections of each of the following courses did you teach this past school year? (Darken one circle on each line.) 

7 or 

0 1 2 3 4 5 6 more 

Physics (e.g.. Regular, Honors, Advanced Placement) 

Physical Science 
Other Science 
Other Non-Science 
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B, Teacher Opinions and Preparedness 



8 . 



In the left section, please rate each of the following in terms of its importance for effective science instruction in the 
grades you teach. In the right section, please indicate how prepared you feel to do each one. (Darken one circle in each 



section on each line.) Importance 

Not Somewhat Fairly Very 

important imnortant imnortant imnortant 


Preparation 

Not Fairly Very 

adequately Somewhat well well 

oreoared oreoared oreoared oreoar 


a. Provide concrete experience before abstract 
concepts. 




b. Develop students' conceptual understanding 
of science. 




c. Take students' prior understanding into 
account when planning curriculum and 
instruction. 




d. Make connections between science and other 
disciplines. 




e. Have students work in cooperative learning 
groups. 








f Have students participate in appropriate 
hands-on activities. 




g. Engage students in inquiry-oriented 
activities. 




h. Have students prepare 

proj ect/laboratory/research reports. 




i. Use calculators. 




j. Use computers. 








k. Engage students in applications of science in 
a variety of contexts. 




1. Use perfonnance-based assessment. 




m. Use portfolios. 




n. Use informal questioning to assess student 
understanding. 




0 . Use calculator/computer-based labs. 




p. Use graphing calculators. 





Please indicate how well prepared you feel to do each of 

Not Fairly Very 

the following. (Darken one circle on each line.) adequately Somewhat well well 

prepared prepared prepared prepared 

a. Lead a class of students using investigative strategies. 

b. Manage a class of students engaged in hands-on/project-based work. 

c. Help students take responsibility for their own learning. 

d. Recognize and respond to student diversity. 

e. Encourage students' interest in science. 

f Use strategies that specifically encourage participation of females and 
minorities in science. 

g. Involve parents in the science education of their students. 
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10 . 



Within science, many teachers feel better prepared to teach some topics than others. How well prepared do you feel to teach 



each of the following topics at the grade levels you teach, whether or not they are currently included in your curriculum? 



(Darken one circle on each line.) 



a. Physics 



Not Fairly Very 

adequately Somewhat well well 

prepared prepared prepared prepared 



1. Kinematics (i.e.. Motion) 

2. Forces (i.e., gravitational, normal, friction, tension) 



3. Newton's Laws 

4. Linear Momentum 



5. Energy (i.e.. Thermodynamics) 

6. Energy as a Societal Issue 

7. Static Electricity 

8. Direct Current Circuits 



b. Scientific methods and inqniry skills 

1. Formulating hypotheses, drawing conclusions, making generalizations 

2. Experimental design 

3. Describing, graphing, and interpreting data 



Questions 11-14 ask about your science teaching. Please answer for your first physics or physical science class of the day 
during this past school year. If you did not teach physics or physical science, please answer for your first science class of the 
day. If you did not teach any science, darken here and skip the remainder of this questionnaire. 



11. What was the subject of this class? Life science/Biology 

(Darken one circle.) Earth/Space science 

Environmental science 

12. What grade level was it? (Darken one circle.) 

Elementary school science Middle school science 



Chemistry Integrated science 

Physics General science 

Physical science 



High school science 



13. About how often did you do each of the following in your science 
instruction in this class? (Darken one circle on each line.) 



Never 



Rarely 
(e.g., a few 
times a 
year) 



Sometimes 
(e.g., once 
or twice 
a month) 



Often 
(e.g., once 
or twice 
a week) 



All or 
almost all 
science 
lessons 




n. Read and comment on the reflections students have written in their 
notebooks or journals. 
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Sometimes Often 
(e.g., once (e.g., once 
or twice or twice 

a month) a week) 

a. Participate in student-led discussions. 

b. Participate in discussions with the teacher to further science 

understanding. 

c. Work in cooperative learning groups. 

d. Make formal presentations to the class. 

e. Read from a science textbook in class. 

f Read other (non-textbook) science-related materials in class. 

g. Answer textbook/worksheet questions. 

h. Review homework/worksheet assignments. 

i. Work on solving a real-world problem. 

j. Share ideas or solve problems with each other in small groups. 

k. Engage in hands-on science activities. 

l. Follow specific instructions in an activity or investigation. 

m. Design or implement their own investigation. 

n. Design objects within constraints (e.g., egg drop, toothpick 

bridge, aluminum boats). 

o. Work on models or simulations. 

p. Work on extended science investigations or projects (a week or 

more in duration). 

q. Participate in field work. 

r. Record, represent, and/or analyze data. 

s. Write reflections in a notebook or journal. 

t. Prepare written science reports. 

u. Use mathematics as a tool in problem-solving. 

V. Use calculators. 

w. Use computers. 

X. Work on portfolios. 

y. Take short-answer tests (e.g., multiple choice, tme/false, 

fill-in-the-blank). 

z. Take tests requiring open-ended responses (e.g., descriptions, 

explanations). 

aa. Engage in performance tasks for assessment purposes. 



14. About how often did students in this class take part in each of the 
following types of activities as part of their science instruction? 
(Darken one circle on each line.) 



Never 



Rarely 
(e.g., a few 
times a 
year) 



THANK YOU!! 

• • • • t 
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